Audio routing for audio-video recording

ABSTRACT

Systems and methods for routing audio for audio-video recordings allow a user to record desired audio with captured video at the time the video is being captured. Audio from one or more sources may be routed to the video capture application and recorded with the video. In one or more examples, audio may be routed from another application, e.g., an audio playback application, running on the same device as the video capture application. In another example, audio may be received from a remote device through a wireless connection. Multiple streams of audio content may be mixed together prior to storing with video. The audio, upon reception, may then be routed to the video capture application for recordation. An audio progression bar may also be provided to indicate duration and elapsed time information associated with the audio being recorded.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. application Ser. No.14,609,423, filed 29 Jan. 2015, which is a continuation of U.S.application Ser. No. 13/424,186, filed 19 Mar. 2012 now U.S. Pat. No.8,977,102, which is a continuation of U.S. application Ser. No.11/612,057, filed 18 Dec. 2006 now U.S. Pat. No. 8,160,421, thedisclosures of which are incorporated herein by reference in theirentirety.

FIELD

The exemplary and non-limiting embodiments disclosed herein relategenerally to a method and system for routing audio for recordation withvideo.

BACKGROUND

For many movies and video clips, audio is often added after the videohas already been captured and recorded. Videos are typically editedusing software that allows a user to incorporate various types of audiocontent (e.g., music, sound effects, narration) into the video. Forexample, a filmmaker may record video for a movie and only later add-inthe sound effects and soundtrack music. This post-recordation editingprocess may be time consuming and laborious as the video and audio mayrequire further editing and/or formatting prior to combining the two.Further, editing software is often needed in addition to the video/audiocapture applications. As such, not only might video and audio editing betime consuming, but it may also increase costs associated with producinga movie or video clip. Additionally, it is often difficult for a usercapturing video to gauge how much audio content is left. Without beingable to determine an amount of audio content remaining, a user may beunable to appropriately time the ending of a video.

SUMMARY

This summary is provided to introduce a selection of concepts in asimplified form that are further described below in the DetailedDescription. The Summary is not intended to identify key features oressential features of the claimed subject matter, nor is it intended tobe used to limit the scope of the claimed subject matter.

Audio routing provides recordation of desired audio content inconjunction with the capture of video. Users may initiate capturing ofvideo and route audio from other sources into the video recording. Forexample, a user may begin recording video using a video captureapplication. The application may then determine whether audio isavailable from one or more sources including other devices and otherapplications. Devices may transfer audio content to other devices usingwireless and wired transmission methods. The application may then promptthe user to decide whether the user wants to route audio content fromthe one or more sources into the video. If so, the audio content may berecorded while the video is being captured. If, however, the user doesnot want to record the audio content with the video, a default videocapture mode may be used. Audio may be routed from other applicationsrunning on the same video capture device or on a different device. Forexample, audio may be transmitted from a first terminal to a secondterminal for recording with video captured by the second terminal. Inanother example, an audio playback application running on a terminal maybe used to provide audio to a video capture application capturing videoon the same terminal.

Audio content from multiple sources may further be mixed prior to beingrecorded with captured video. For example, if audio content is availablefrom both a remote terminal and an input device of a video capturingterminal, the audio content streams from both sources may be combinedusing various mixing methods to produce a composite audio signal. Theaudio signal may then be encoded, synchronized with and/or recorded withthe video being captured by the capturing terminal. Alternatively oradditionally, a user may select one or more streams of audio contentfrom multiple available streams to mix and record with captured video.Further, users may be provided mixing controls to adjust variousattributes of the input streams and/or of the resultant signal. Forexample, a user may configure a recording such that background noisereceived from a microphone input device is recorded at a lower volumethan sound effects received from another device. In another example, auser may modify the treble and bass levels of a resultant compositeaudio signal.

According to another aspect, an audio progress bar may be displayed on auser interface of a video capturing application. The audio progress barmay provide a variety of information including audio content duration,elapsed playback time, amount downloaded/received, audio content titleand the like. The audio progress bar allows a user to gauge an amount ofremaining audio. As such, a user who has thirty seconds of remainingaudio may determine whether to speed up or slow down his or her videocapturing.

BRIEF DESCRIPTION OF THE DRAWINGS

The foregoing summary of the disclosed embodiments, as well as thefollowing detailed description of illustrative embodiments, is betterunderstood when read in conjunction with the accompanying drawings,which are included by way of example, and not by way of limitation withregard to the disclosed embodiments:

FIG. 1 illustrates a mobile terminal on which one or more aspectsdescribed herein may be implemented.

FIG. 2 is a flowchart illustrating a method for capturing video with afirst application and recording the captured video along with audioreceived from a second application according to one or more aspectsdescribed herein.

FIGS. 3A and 3B illustrate block diagrams corresponding to systems forcapturing video with audio from one or more sources according to one ormore aspects described herein.

FIG. 4 is a flowchart illustrating a method for capturing video using afirst device and recording the video with audio from a second deviceaccording to one or more aspects described herein.

FIGS. 5A and 5B illustrate video capture user interfaces displaying anaudio progression bar and mixer controls according to one or moreaspects described herein.

DETAILED DESCRIPTION

In the following description of various illustrative embodiments,reference is made to the accompanying drawings, which form a parthereof, and in which is shown, by way of illustration, variousembodiments in which the disclosed embodiments may be practiced. It isto be understood that other embodiments may be utilized and structuraland functional modifications may be made without departing from thescope of the disclosed embodiments.

FIG. 1 illustrates a block diagram of a terminal including processor 128connected to user interface 130, memory 134 and/or other storage, anddisplay 136. Mobile terminal 112 may also include battery 150, speaker153 and antennas 154. User interface 130 may further include a keypad,touch screen, voice interface, one or more arrow keys, joy-stick, dataglove, mouse, roller ball, touch screen, or the like. Mobile terminal112 may comprise a computer, personal data assistant (PDA), mobiletelephone and the like.

Computer executable instructions and data used by processor 128 andother components within mobile terminal 112 may be stored in a computerreadable memory 134. The memory may be implemented with any combinationof read only memory modules or random access memory modules, optionallyincluding both volatile and nonvolatile memory. Software 140 may bestored within memory 134 and/or storage to provide instructions toprocessor 128 for enabling mobile terminal 112 to perform variousfunctions. Alternatively, some or all of mobile device 112 computerexecutable instructions may be embodied in hardware or firmware (notshown).

Mobile terminal 112 may be configured to receive, decode and processdigital broadband broadcast transmissions that are based, for example,on the DVB standard, through a specific DVB receiver 141. The mobiledevice may also be provided with other types of receivers for digitalbroadband broadcast transmissions. Additionally, mobile terminal 112 mayalso be configured to receive, decode and process transmissions throughFM/AM Radio receiver 142, WLAN transceiver 143, and telecommunicationstransceiver 144. Transceivers 143 and 144 may, alternatively, beseparated into individual transmitter and receiver components (notshown). In one aspect of the disclosed embodiments, mobile terminal 112may receive Radio Data System (RDS) messages. Other transmission andreception systems may also be used including Bluetooth transceivers. Inone or more instances, signals may be transmitted to and received fromanother mobile terminal (not shown). For example, audio, video and othersignals may be transmitted between two terminals using varioustransmissions protocols (e.g., WLAN or Bluetooth).

Additionally, mobile terminal 112 may comprise various multimediacapture systems such as video/image capture system 170 and audio capturesystem 175. Video/image capture system 170 may comprise a video capturedevice capable of capturing still images and motion video (e.g., a videocamera). For example, a cell phone may include a camcorder feature thatallows a user to record live video. Audio capture system 175 may includedevices such as a microphone, an electronic instrument, a playbackdevice and the like. The video capture and audio capture devices may beexternal to mobile terminal 112 or built-in internal devices. Forexample, an electronic instrument may be a plug-in device connectable toterminal 112 through a jack or other connectors. Alternatively oradditionally, a microphone may be built-in to terminal 112. In one ormore configurations, systems 170 and 175 may further include adaptersfor processing the signals received from the capture devices. Each ofvideo capture system 170 and audio capture system 175 may be connectedto memory 140 and processor 128. Captured audio and video may be storedin memory 140 while processor 128 may execute instructions orapplications used in audio and video capturing. For example, software134 may include one or more applications that manages capture systems170 and 175 and their captured content.

FIG. 2 is a flowchart illustrating a method for receiving and recordingaudio in conjunction with captured video. In step 200, a video captureapplication running on a terminal (e.g., mobile terminal 112 of FIG. 1)may receive a command to begin capturing video (e.g., live video) usinga video capture device. The video capture application may then determinewhether audio is available in step 205. The determination may includethe video capture application scanning each of the terminal's audioinputs and outputs to determine whether audio is being inputted into oroutputted from the device. In one example, the video capture applicationmay determine whether audio content is currently being played andoutputted from the terminal and/or whether audio is being received fromanother terminal or device. In one or more configurations, a defaultmusic track may be played automatically upon initiating video capture.

In response to determining that audio is being inputted into oroutputted from the terminal, the video capture application may ask theuser whether he or she wants to add the corresponding audio into thevideo recording in step 210. If the application determines that the userwould like to add the audio into the video recording in step 215, theapplication may begin capturing the audio in step 220. In step 223, theapplication may display an audio progression bar during the capturing ofthe audio and video. The progression bar may indicate a variety of audioinformation such as audio length, elapsed time and content size. Thecaptured audio may then be encoded and stored along with the videorecording in one or more multimedia formats in step 225. Multimediaformats may include Motion Picture Experts Group (MPEG) format,QuickTime formats and the like. If, however, in steps 210 and 215, it isdetermined that the user does not want to add the available audio or ifaudio is not available, only the video may be recorded and stored instep 230.

In one or more configurations, if audio is detected from multiple inputsand/or outputs, the capture application may ask a user whether he or shewould like to mix or combine the audio from two or more of the variousinputs and outputs. Alternatively or additionally, a user may be given achoice of audio content to record with the captured video. For example,a capture application may display the various audio content available(e.g., being inputted to or outputted from a terminal on which theapplication is running) and allow the user to individually select one ormore of the audio content for mixing and recording.

FIGS. 3A and 3B illustrate block diagrams of systems for capturing videoalong with audio from different sources. FIG. 3A illustrates a systemfor capturing video using first application 305 on terminal 301 andrecording audio output from second application 310 along with thecaptured video. First application 305 may, for example, include anaudio-video capture program configured to record motion video and audio.Second application 310, on the other hand, may include an audio playbackapplication such as WINDOWS MEDIA PLAYER, REALPLAYER, WINAMP and thelike. Such audio playback applications may output a variety of audiocontent including stored audio files (e.g., from storage 330), broadcastaudio (e.g., radio) and the like. Audio output application 310 may beoutputting audio in the background while capture application 305 isrecording video and/or audio. As such, the system may allow a user toroute the audio being outputted by application 310 to captureapplication 305 so that the outputted audio is incorporated into acaptured audio-video file. In one or more configurations, the audiosignals being outputted by application 310 may be duplicated and routedto application 305 as audio input. The routing may be performed by theaudio output application, the video capture application or anintermediate application or component (not shown) of terminal 301.Application 305 may further combine multiple audio streams (e.g., frommultiple sources) prior to recording and associating the plurality ofaudio with the captured video. Various methods for mixing audio inputmay be used to produce a composite audio signal for recording withcaptured video. Additionally, captured video may be displayed on display315 while recording. Other information may also be displayed on display315 including information relating to audio being recorded with thevideo. The recorded audio and video may be stored in storage 330.

According to one or more aspects, terminal 301 may further include audiomixer 320 that allows multiple streams of audio to be mixed or combined.Thus, in one example, background music received from audio playbackapplication 310 may be mixed with narration received from external audiosystem 325 (e.g., a microphone) connected to terminal 301. Particularly,the two streams of audio content may be mixed to form a single audiosignal for recording with the video. Alternatively or additionally,terminal 301 may receive radio and other audio broadcasts throughbroadcast receiver 335. The audio content received through receiver 335may similarly be mixed into a composite audio signal to be recorded withthe captured video. In one or more configurations, audio mixer 320 mayallow a user to control the volume and/or other attributes of each audiostream. As such, background music may be recorded at a lower volume thannarration received through external audio system 325.

FIG. 3B illustrates a system for capturing video using captureapplication 350 on terminal 355 in conjunction with audio transmitted toterminal 355 from terminal 356. Audio may be transferred from terminal356 to terminal 355 using a variety of content transfer applications,such as applications 360 and 361. As such, before or after application350 begins capturing video, terminal 356 may request and initiate atransfer of audio content to terminal 355 using communicationapplication 361. Communication applications 360 and 361 may includeBluetooth and/or WLAN communication protocols and/or software forcontrolling the receivers, transmitters and/or transceivers of terminals355 and 356, respectively. The audio content to be transferred may beextracted from storage 365 in terminal 356 and subsequently transmittedby application 361. If the transfer is accepted by terminal 355, theaudio content may be downloaded into storage medium 380 of terminal 355.In one or more configurations, the transfer of audio content comprises astreaming transfer. The transmitted audio content may further be routedfrom communication application 360 to video capture application 350 suchthat the received audio is recorded along with the captured video. Theaudio-video recording may be stored in storage medium 380.

As discussed, if different and additional audio content is availablefrom another source, the additional audio content may be mixed togetherto form a single audio signal for recording and synchronization with thevideo. For example, audio content may be received from each of device356, external audio system 375 and broadcast receiver 385. Each streamof audio may be fed into audio mixer 370 that combines the audio signalsinto a single composite signal for recordation with video captured byapplication 350. Mixer 370 may provider controls for a user to adjustvarious audio characteristics of each audio content stream for mixing.

FIG. 4 is a flowchart illustrating a method for recording, at a firstdevice, audio received from another device along with captured video. Instep 400, a video capture application of a first device may initiatevideo capturing in response to user input. For example, a user mayselect a record option that activates video capturing. In step 405, thevideo capture application may determine whether audio is beingtransmitted from a second device. For example, a user may be downloadingor otherwise receiving a music file from a friend's mobile terminal. Inresponse to determining that audio is being transmitted from a seconddevice, the video capture application may determine whether the userwould like the audio being transmitted to be recorded as part of thevideo in step 410. If the application determines that the user wants theaudio to be recorded, the application may request that the audio berouted to the application in step 415. For example, the application maysend a request to the operating system for the audio content. Inresponse to the request, the application may begin receiving the audiocontent in step 420.

In step 425, the application may further determine whether other audiocontent is being captured or otherwise available. If other audio isavailable and/or being captured, the audio content transmitted from thesecond device may be mixed with the other audio content in step 430. Inone or more configurations, a user may be asked whether the audiocontent should be mixed. If the user does not want the audio contentmixed, he or she may be asked to select the audio content to berecorded. In addition, a user may be provided controls for configuringthe output levels of the various audio content streams to theaudio-video recording. The controls may also allow a user to adjustvarious audio attributes such as treble and bass. In step 435, the audiocontent (if no other audio content is being captured) or mixed audiocontent may then be synchronized and recorded with the captured video.If, on the other hand, audio is not being received from another device,the captured video may be encoded and stored alone or with audio contentfrom one or more other sources in step 440.

FIG. 5A illustrates a video capture user interface displaying an audioprogress bar. User interface 500 may display video 505 that is currentlybeing captured along with one or more informational overlays 510,515 and520. Overlay 510 may provide capture specifications including captureresolution, frame rate and audio sampling rate. Overlay 515, on theother hand, may display metadata information such as author information,title information, current file size and date. Overlay 520 may displayone or more controls 525 for managing the recordation of the video andaudio. Controls 525 may include stop, rewind, pause and the like.Controls 525 may control both the video capture and audio recording oreach may be controlled separately. Overlay 520 may further include audioprogress bar 530 that indicates duration 535 of the corresponding audiocontent, current position 540 in the audio content and elapsed playbacktime 545. Alternatively or additionally, progress bar 530 may furtherindicate an amount downloaded or transferred if the content is beingreceived from another device. In one instance, the amount downloaded ortransferred may be indicated by a first bar of a first color while anelapsed playback time may be indicated by a second bar of a second colorto aid in differentiation. If audio is being received and mixed frommultiple sources, an audio progression bar for each audio stream may bedisplayed in interface 500.

In one or more configurations, audio progress bar 530 may be displayedon a different device, e.g., the device transmitting the audio content.Alternatively or additionally, progress bar may be displayed on a seconddisplay device. Displaying bar 530 on another device or display mayconserve space on a user interface. Video capture interface 505 mayfurther include a mixer 550 option for mixing one or more audio streams.Mixer option 550 might only be displayed if more than one audio streamis available for recording.

FIG. 5B illustrates user interface 505 after mixer option 550 of FIG. 5Ahas been selected. Selection of mixer option 550 (FIG. 5A) opens mixerinterface 560 displaying multiple sets of controls 580, 581, 582 and 583for audio inputs and audio attributes (e.g., treble and bass). The mixerthus may allow a user to control the level at which individual audiocontent streams are recorded. For example, a user may set narrationreceived through a microphone to record louder than background musicfrom a broadcast receiver. Each set of controls 580, 581, 582 and 583may include a left channel control, e.g., control 585, and a rightchannel control, e.g., control 586. As such, volume levels for audiooutput through the different individual channels may be controlledseparately and independently. A mixer control such as control 585 mayinclude control bar 575 that conveys a range along which control 585 maybe adjusted. Slider 570 may be displayed in bar 575 to indicate acurrent level at which the control is set. A user may adjust theposition of slider 575 to a desired level in bar 575. Other controls andoptions may be included in mixer interface 560 including controls foractivating and deactivating audio inputs.

According to one or more configurations, video being captured and itsassociated audio and video information may be displayed in differentportions of the user interface. For example, audio information may bedisplayed in a bar positioned across the bottom of the interface whilevideo information may be displayed in a bar vertically positioned alongthe left side of the interface. The video being captured may occupy theremainder of the interface. The various interface configurations may bemodified according to user preferences or based on an optimal viewingsize of the video being captured.

Additionally, the methods and features recited herein may further beimplemented through any number of computer readable mediums that areable to store computer readable instructions. Examples of computerreadable media that may be used include RAM, ROM, EEPROM, flash memoryor other memory technology, CD-ROM, DVD or other optical disk storage,magnetic cassettes, magnetic tape, magnetic storage and the like.

While illustrative systems and methods as described herein embodyingvarious aspects of the disclosed embodiments are shown, it will beunderstood by those skilled in the art, that the disclosed aspects arenot limited to these embodiments. Modifications may be made by thoseskilled in the art, particularly in light of the foregoing teachings.For example, each of the elements of the aforementioned embodiments maybe utilized alone or in combination or subcombination with elements ofthe other embodiments. It will also be appreciated and understood thatmodifications may be made without departing from the true spirit andscope of the disclosed embodiments. The description is thus to beregarded as illustrative instead of restrictive of the disclosedembodiments.

The invention claimed is:
 1. A method comprising: capturing videocontent at a device via a video capture application on the device;receiving external audio at an audio source of the device; receivingaudio content at the device via an audio playback application on thedevice; combining the audio content and external audio into a combinedaudio signal; and recording, by the video capture application, thecombined audio signal with the video content into a single audio-videofile.
 2. The method of claim 1, further comprising: routing the audiocontent from the audio playback application to an audio mixer at thedevice for combining with the external audio; and wherein the combiningstep is performed by the audio mixer.
 3. The method of claim 2, whereinthe routing step is performed by an intermediate application at thedevice.
 4. The method of claim 2, further comprising: adjusting, by theaudio mixer, relative volumes of the audio content and external audioresponsive to user input.
 5. The method of claim 1, wherein the audiosource is a microphone; and wherein the video capture device is a cameraat the apparatus.
 6. The method of claim 1, further comprising:displaying, via the video capture application while capturing the videocontent, one or more available audio sources for selection by a user,the one or more available audio sources including the combined audiosignal.
 7. An apparatus, comprising: a processor; a video capturedevice; an audio capture device; and one or more non-transitory computerreadable media storing program instructions comprising a video captureapplication and an audio playback application, wherein the instructions,when executed by the processor, cause the apparatus to at least: capturevideo content from the video capture device via the video captureapplication on the device; capture external audio from the audio capturedevice; receive audio content from the audio capture device via theaudio playback application; combine the audio content and external audiointo a combined audio signal; and record, via the video captureapplication, the combined audio signal with the video content into asingle audio-video file.
 8. The apparatus of claim 7, wherein the storedprogram instructions further comprise an audio mixer; and wherein theinstructions are configured to further cause the apparatus to: route theaudio content from the audio playback application to the audio mixer forcombining with the external audio; and wherein the combining of theaudio content and the external audio is performed via the audio mixer.9. The apparatus of claim 8, wherein the stored program instructionsfurther comprise an intermediate application; and wherein the routing ofthe audio content is performed via the intermediate application.
 10. Theapparatus of claim 8, wherein the instructions are configured to furthercause the apparatus to: adjust, via the audio mixer, relative volumes ofthe audio content and external audio responsive to user input.
 11. Theapparatus of claim 7, wherein the audio capture device is a microphoneat the apparatus; and wherein the video capture device is a camera atthe apparatus.
 12. The apparatus of claim 7, further comprising: adisplay; and wherein the instructions are configured to further causethe apparatus to: present at the display, via the video captureapplication while capturing the video content, one or more availableaudio sources for selection by a user, the one or more available audiosources including the combined audio signal.
 13. An apparatus,comprising: a processor; a video capture device; and one or morenon-transitory computer readable media storing program instructionscomprising a video capture application, an audio playback application,and an intermediate application, wherein the instructions, when executedby the processor, cause the apparatus to at least: capture video contentfrom the video capture device via the video capture application on thedevice; receive an audio content via the audio playback application;route the audio content from the audio playback application to the videocapture application via the intermediate application; and record, viathe video capture application, the audio content with the video contentinto a single audio-video file.
 14. The apparatus of claim 13, furthercomprising: an audio capture device; wherein the instructions areconfigured to further cause the apparatus to: capture external audiofrom the audio capture device; combine the audio content and externalaudio into a combined audio signal; and wherein the video captureapplication records the combined audio signal with the video contentinto a single audio-video file.
 15. The apparatus of claim 14, whereinthe stored program instructions further comprise an audio mixer; whereincombining of the audio content and the external audio is performed viathe audio mixer.
 16. The apparatus of claim 14, wherein the instructionsare configured to further cause the apparatus to: adjust, via the audiomixer, relative volumes of the audio content and external audioresponsive to user input.
 17. The apparatus of claim 14, wherein theaudio capture device is a microphone at the apparatus; and wherein thevideo capture device is a camera at the apparatus.
 18. A methodcomprising: capturing video content at a device via a video captureapplication on the device; receiving an audio content at the device viaan audio playback application on the device; routing the audio contentto the video capture application via an intermediate application on thedevice; and recording, by the video capture application, the audiocontent with the video content into a single audio-video file.
 19. Themethod of claim 18, further comprising: capturing external audio at anaudio source of the device; combining the audio content and externalaudio into a combined audio signal; wherein the recording step recordsthe combined audio signal with the video content into a singleaudio-video file.
 20. The method of claim 19, further comprising:routing the audio content from the audio playback application to anaudio mixer on the device for combining with the external audio; whereinthe combining step is performed by the audio mixer.
 21. The method ofclaim 20, further comprising: adjusting, by the audio mixer, relativevolumes of the audio content and external audio responsive to userinput.
 22. The method of claim 19, wherein the audio source is amicrophone; and wherein the video capture device is a camera at theapparatus.